Decision markets

Decision markets are systems in which participants buy and sell contracts tied to future outcomes, allowing prices to express a collective forecast. Their significance is not that markets are infallible, or that wagering is the only way to gather judgments, but that a properly designed mechanism can combine information held by many people—including information no single participant possesses in full. The book treats election markets, entertainment forecasts, sports betting, and proposed intelligence markets as variations on this central idea, while stressing that their accuracy depends on the conditions under which people contribute and their judgments are combined.

How the Market Turns Judgments into Forecasts

The Iowa Electronic Markets (IEM), founded in 1988 and run by the College of Business at the University of Iowa, lets participants trade futures contracts tied to elections. One common contract pays a dollar if a named candidate wins and nothing if the candidate loses. Its price therefore serves as an estimate of the chance of victory: a price of 50 cents corresponds roughly to a 50 percent probability. A second contract pays according to a candidate’s eventual share of the popular vote. If a contract on George W. Bush had been bought in 2000, it would have paid 48 cents when he received 48 percent of the vote.

The relationship between price and outcome makes the market’s performance assessable. In a study covering 49 elections from 1988 through 2000, election-eve IEM prices missed the final result by an average of 1.37 percentage points in presidential elections, 3.43 points in other U.S. elections, and 2.12 points in foreign elections. The market generally outperformed major national polls and was more accurate than them even months before voting. Across the presidential elections in that period, researchers examined 596 polls; on the day each poll was released, the IEM price was more accurate three-quarters of the time.

Those prices changed as new information arrived, but were considerably less volatile than poll results, which could swing sharply. The IEM was also small—never more than about 800 traders—and its participants did not resemble the electorate: most were men, and a disproportionate, though shrinking, share lived in Iowa. They were not simply predicting their own voting behavior. Yet their market forecasts proved better than forecasts obtained by asking voters what they themselves intended to do. The contrast suggests that a prediction market can draw on information beyond the participants’ direct representation of the population being forecast.

Accuracy Across Different Markets

Sports betting provides another model of price-based aggregation. At the Mirage, bookmaker Robert Walker adjusted the point spread when wagers indicated that a line might be wrong. Before the 2001 Super Bowl, the Baltimore Ravens opened as 2.5-point favorites; after the casino received two early $3,000 bets on Baltimore, Walker raised the spread to three. The line was initially set by the bookmaker, but thereafter shifted in response to bettors’ choices. In the final form, it represented the bettors’ combined judgment of the likely result.

The spread was not a crystal ball: it only weakly predicted the final scores of most NFL games. But gamblers found it difficult to beat consistently. Favorites covered in about half of games and underdogs in the other half, with no obvious recurring error such as home teams winning more often than the crowd expected. A small historical exception appeared in weeks 15, 16, and 17, when home underdogs had been good bets. Overall, the Mirage’s closing line was the most reliable available forecast of NFL outcomes about three-quarters of the time. Studies of other major sports likewise found markets generally efficient; horse-racing odds, for example, reliably ranked horses by likelihood of winning and gave reasonably good estimates of their chances.

The Hollywood Stock Exchange (HSX) extended the idea to entertainment. It lets users wager on box-office returns, opening-weekend performance, and award outcomes. In March 2000, a team of 12 reporters from The Wall Street Journal canvassed Academy members about their Oscar ballots. Despite the Academy’s objection and its request that members not speak to reporters, 356 members—about 6 percent—disclosed their votes on condition of anonymity. The newspaper correctly predicted five of six major awards. The HSX predicted all six. In 2002, it also selected 35 of the eventual 40 Oscar nominees. Its box-office forecasts were less accurate than the IEM’s election forecasts, but a comparison by Harvard Business School professor Anita Elberse found that the HSX closing price on the night before a film opened was the best available forecast of its weekend box office. The exchange’s owner, Cantor Index Holdings, began marketing the data to Hollywood studios.

These examples do not establish that every market predicts every outcome well. Rather, they illustrate how a price can function as a shared estimate whose quality can be compared with later results. A market price does not need to be the opinion of a representative electorate or a panel of recognized specialists to carry information; the cases show that participants’ combined judgments can be useful even when the group is relatively small or socially unrepresentative.

Incentives, Participation, and Limits

Financial stakes may help participants concentrate. The IEM uses real money, but a trader can invest no more than $500, and the average trader has only $50 at stake. HSX wagering, by contrast, uses play money. David Pennock, who studied these markets, found that active traders could nevertheless devote serious time and effort to them because status and reputation supplied incentives. The comparison suggests that cash is not the sole reason participants engage, even if financial rewards may help explain why IEM forecasts tend to be more accurate.

By the fall of 2003, the range of proposed market subjects had expanded substantially. On NewsFutures and TradeSports, participants could wager on whether Kobe Bryant would be convicted of sexual assault, whether and when weapons of mass destruction would be found in Iraq, and whether Ariel Sharon would remain in power longer than Yasir Arafat. At UCLA, Ely Dahan experimented with a classroom market in which students traded securities representing consumer goods and services, including SUVs, ski resorts, and personal digital assistants. The forecasts resembled those of conventional market research, at much lower cost. MIT’s Technology Review also created Innovation Futures, where participants could wager on technological developments. Robin Hanson, an early advocate of decision markets, proposed applying them to scientific research and government policy.

The range of possible subjects does not guarantee that all such markets will work. A market may attract too few participants to produce an intelligent forecast, or attempt to predict something inherently unpredictable. The book’s claim is conditional: where a problem suits the method and the necessary conditions are present, diversity, independence, and decentralization can support good collective judgment. Markets make it relatively quick to turn many separate opinions into one estimate, but the estimate’s apparent precision should not be mistaken for certainty.

The Conditions That Make a Crowd Useful

The framework for evaluating decision markets is the broader account of wise crowds. A group needs diversity of opinion, so participants bring some private information or a distinctive interpretation; independence, so judgments are not simply copied from others; decentralization, so people can draw on specialized or local knowledge; and aggregation, so individual assessments can become a collective decision. With enough diverse and independent estimates, individual errors can cancel. But cancellation alone does not make a group knowledgeable: there must be relevant information in the estimates to begin with. A crowd can be accurate when its members collectively hold a substantial picture of the world, not merely because many people have made guesses.

The book contrasts this with the danger of shared signals that pull judgments in the same direction. In the Caltech bubble experiment, students traded shares over fifteen five-minute periods. Each share paid a 24-cent dividend per period, so its total value over the experiment was known in advance: at most $3.60 before the first dividend, then $3.36, $3.12, and successively less as periods passed. Despite that transparency, the trading price jumped to $3.50 and remained there almost to the end. Even when the shares were worth less than a dollar in remaining dividends, participants still paid $3.50, expecting to resell to someone else. The experiment demonstrates that exchange and a visible price do not automatically produce sound judgment; participants can instead follow one another’s expectations.

The book makes the same point about financial news. During the late-1990s stock-market bubble, CNBC’s audience reached seven million viewers a week, while the network’s nonstop market coverage and continuously running ticker made investors’ attention to one another more immediate. In a study of CNBC’s Midday Call, stock prices rose within the first 15 seconds of a positive report, and trading volume sextupled during the first minute. The speed of the response suggested that investors were reacting not only to what the report said, but to the expectation that others would trade on it. Shared attention can therefore weaken independence: information about what other traders think becomes a cue to imitate them, and the market’s collective judgment may deteriorate even as information becomes more abundant.

The Policy Analysis Market

The clearest political test of the idea was FutureMAP, an abortive project funded by the Defense Advanced Research Projects Agency. It proposed two kinds of markets. Small internal markets, perhaps involving 20 or 30 intelligence analysts and outside experts, would let participants trade contracts based on expectations about events in the Middle East and elsewhere. Because analysts might use classified information and intelligence data, these markets could have addressed questions such as the probability of particular terrorist attacks. The rationale was to offer a way around bureaucratic rivalry and political pressure: participants would have a reason to make accurate forecasts rather than to please a supervisor, an agency, or the White House. Bringing analysts from different organizations together might also have produced a collective judgment that the intelligence system had struggled to reach.

The second proposal, the Policy Analysis Market (PAM), would have been open to the public. Its basic premise was that a market focused on the Middle East might supply intelligence that conventional channels missed, just as other markets had shown forecasting value. Public participation could bring in information from people whom the intelligence community might not otherwise hear, including people deterred by patriotism, fear, or resentment. The proposal was sharply attacked in the summer of 2003. Senators Ron Wyden and Byron Dorgan called it “harebrained,” “offensive,” and “useless”; Wyden also dismissed it as a “fairy tale.” The public response, among those who learned of the plan, was similarly hostile, and the project was killed.

The objections focused in part on the prospect of ordinary people profiting from catastrophes. The book acknowledges the visceral discomfort of betting on an assassination attempt, but distinguishes the forecasting method from the moral character of the event being forecast. Government analysts already ask questions about the stability of Jordan, the possible fall of the House of Saud, and who might lead the Palestinian Authority. PAM was proposed as an additional way of gathering information, not as a replacement for traditional intelligence. If the markets failed, their forecasts could have shown that; cancellation ensured that their possible contribution would remain unknown. The dispute thus exposed a divide between a mechanism’s potential information value and the ethical or political acceptability of using it.

Significance and Scope

Decision markets are one among several ways to aggregate judgments. Stock prices, votes, sports point spreads, pari-mutuel odds, computer algorithms, and futures contracts all attempt to turn dispersed information into a collective assessment. The book does not argue that a futures market is inherently smarter than a search algorithm or a betting pool. What matters is whether the arrangement preserves the conditions that make a crowd capable of combining what its members know.

That distinction is especially important for organizations. A market can, in principle, collect information from varied sources, assess probabilities, and produce an estimate about an uncertain future—tasks central to corporate strategy. Yet companies have largely remained indifferent to using employees’ collective judgments this way. A useful mechanism does not ensure that institutions will adopt it, or that they will welcome a result that challenges established authority. Decision markets therefore stand both as practical forecasting tools and as a challenge to the assumption that good decisions must come from a single expert or a centralized chain of command.